Unitary Representations of Lie Groups with Reflection Symmetry

We consider the following class of unitary representations π of some (real) Lie group G which has a matched pair of symmetries described as follows: (i) Suppose G has a period-2 automorphism τ , and that the Hilbert space H(π) carries a unitary operator J such that Jπ = (π ◦ τ)J (i.e., selfsimilarity). (ii) An added symmetry is implied if H(π) further contains a closed subspace K0 having a certain order-covariance property, and satisfying the K0-restricted positivity: 〈v Jv〉 ≥ 0, ∀v ∈ K0, where 〈· ·〉 is the inner product in H(π). From (i)–(ii), we get an induced dual representation of an associated dual group Gc. All three properties, selfsimilarity, order-covariance, and positivity, are satisfied in a natural context when G is semisimple and hermitean; but when G is the (ax + b)-group, or the Heisenberg group, positivity is incompatible with the other two axioms for the infinitedimensional irreducible representations. We describe a class of G, containing the latter two, which admits a classification of the possible spaces K0 ⊂ H(π) satisfying the axioms of selfsimilarity and order-covariance.